Computer Aided Skin Lesion Diagnosis with Humans in the Loop

Despite much progress made in recent years, computer is still incapable of reliably and accurately recognising images of most real world problems, including images of skin diseases. In this paper, we have developed an interactive skin lesion recognition system based on a human in the loop visual recognition technology, where computer vision algorithms and models of human responses to a series of simple perceptual questions are combined together to achieve very high recognition rates (over 96%). We have designed the first ever dermatology “Question and Answer” bank consisting of 21 questions and over 100 possible answers that can be effectively used in a human in the loop skin lesion recognition system. We present experimental results to show that for some diseases, computer vision technique can only achieve a recognition rate of 20%, while with human in the loop the performance can be boosted to over 96%. We also show that users do not require any medical knowledge to answer these questions to achieve excellent recognition rates.